Exploring the Memory-Bandwidth Tradeoff in an Information-Centric Network

An information-centric network should realize significant economies by exploiting a favourable memory-bandwidth tradeoff: it is cheaper to store copies of popular content close to users than to fetch them repeatedly over the Internet. We evaluate this tradeoff for some simple cache network structures under realistic assumptions concerning the size of the content catalogue and its popularity distribution. Derived cost formulas reveal the relative impact of various cost, traffic and capacity parameters, allowing an appraisal of possible future network architectures. Our results suggest it probably makes more sense to envisage the future Internet as a loosely interconnected set of local data centers than a network like today's with routers augmented by limited capacity content stores.Comment: Proceedings of ITC 25 (International Teletraffic Congress), Shanghai, September, 201

Impact of traffic mix on caching performance in a content-centric network

For a realistic traffic mix, we evaluate the hit rates attained in a two-layer cache hierarchy designed to reduce Internet bandwidth requirements. The model identifies four main types of content, web, file sharing, user generated content and video on demand, distinguished in terms of their traffic shares, their population and object sizes and their popularity distributions. Results demonstrate that caching VoD in access routers offers a highly favorable bandwidth memory tradeoff but that the other types of content would likely be more efficiently handled in very large capacity storage devices in the core. Evaluations are based on a simple approximation for LRU cache performance that proves highly accurate in relevant configurations

Flow-Aware traffic control for a content-centric network,

International audienc

SuperConText: Supervised Contrastive Learning Framework for Textual Representations

In the last decade, Deep neural networks (DNNs) have been proven to outperform conventional machine learning models in supervised learning tasks. Most of these models are typically optimized by minimizing the well-known Cross-Entropy objective function. The latter, however, has a number of drawbacks, including poor margins and instability. Taking inspiration from the recent self-supervised Contrastive representation learning approaches, we introduce Supervised Contrastive learning framework for Textual representations (SuperConText) to address those issues. We pretrain a neural network by minimizing a novel fully-supervised contrastive loss. The goal is to increase both inter-class separability and intra-class compactness of the embeddings in the latent space. Examples belonging to the same class are regarded as positive pairs, while examples belonging to different classes are considered negatives. Further, we propose a simple yet effective method for selecting hard negatives during the training phase. In extensive series of experiments, we study the impact of a number of parameters on the quality of the learned representations (e.g. the batch size). Simulation results show that the proposed solution outperforms several competing approaches on various large-scale text classification benchmarks without requiring specialized architectures, data augmentations, memory banks, or additional unsupervised data. For instance, we achieved top-1 accuracy of 61.94% on the Amazon-F dataset, which is 3.54% above the best result obtained when using the cross-entropy with the same model architecture

SuperConText: Supervised Contrastive Learning Framework for Textual representations

International audienceIn the last decade, Deep neural networks (DNNs) have been proven to outperform conventional machine learning models in supervised learning tasks. Most of these models are typically optimized by minimizing the well-known Cross-Entropy objective function. The latter, however, has a number of drawbacks, including poor margins and instability. Taking inspiration from the recent selfsupervised Contrastive representation learning approaches, we introduce Supervised Contrastive learning framework for Textual representations (SuperConText) to address those issues. We pretrain a neural network by minimizing a novel fully-supervised contrastive loss. The goal is to increase both inter-class separability and intra-class compactness of the embeddings in the latent space. Examples belonging to the same class are regarded as positive pairs, while examples belonging to different classes are considered negatives. Further, we propose a simple yet effective method for selecting hard negatives during the training phase. In extensive series of experiments, we study the impact of a number of parameters on the quality of the learned representations (e.g. the batch size). Simulation results show that the proposed solution outperforms several competing approaches on various large-scale text classification benchmarks without requiring specialized architectures, data augmentations, memory banks, or additional unsupervised data. For instance, we achieved top-1 accuracy of 61.94% on the Amazon-F dataset, which is 3.54% above the best result obtained when using the cross-entropy with the same model architecture

Improving Machine Translation of Arabic Dialects through Multi-Task Learning

International audienceNeural Machine Translation (NMT) systems have been shown to perform impressively on many language pairs compared to Statistical Machine Translation (SMT). However, these systems are data-intensive, which is problematic for the majority of language pairs, and especially for low-resource languages. In this work, we address this issue in the case of certain Arabic dialects, those variants of Modern Standard Arabic (MSA) that are spelling non-standard, morphologically rich, and yet resource-poor variants. Here, we have experimented with several multitasking learning strategies to take advantage of the relationships between these dialects. Despite the simplicity of this idea, empirical results show that several multitasking learning strategies are capable of achieving remarkable performance compared to statistical machine translation. For instance, we obtained the BLUE scores for the Algerian → Modern-Standard-Arabic and the Moroccan → Palestinian of 35.06 and 27.55, respectively, while the scores obtained with a statistical method are 15.1 and 18.91 respectively. We show that on 42 machine translation experiments, and despite the use of a small corpus, multitasking learning achieves better performance than statistical machine translation in 88% of cases

SimSCL: A Simple fully-Supervised Contrastive Learning Framework for Text Representation

International audienceDuring the last few years, deep supervised learning models have been shown to achieve state-of-the-art results for Natural Language Processing tasks. Most of these models are trained by minimizing the commonly used cross-entropy loss. However, the latter may suffer from several shortcomings such as sub-optimal generalization and unstable fine-tuning. Inspired by the recent works on self-supervised contrastive representation learning, we present SimSCL, a framework for binary text classification task that relies on two simple concepts: (i) Sampling positive and negative examples given an anchor by considering that sentences belonging to the same class as the anchor as positive examples and samples belonging to a different class as negative examples and (ii) Using a novel fully-supervised contrastive loss that enforces more compact clustering by leveraging label information more effectively. The experimental results show that our framework outperforms the standard cross-entropy loss in several benchmark datasets. Further experiments on Moroccan and Algerian dialects demonstrate that our framework also works well for under-resource languages

Real-Time Infoveillance of Moroccan Social Media Users’ Sentiments towards the COVID-19 Pandemic and Its Management

The impact of COVID-19 on socio-economic fronts, public health related aspects and human interactions is undeniable. Amidst the social distancing protocols and the stay-at-home regulations imposed in several countries, citizens took to social media to cope with the emotional turmoil of the pandemic and respond to government issued regulations. In order to uncover the collective emotional response of Moroccan citizens to this pandemic and its effects, we use topic modeling to identify the most dominant COVID-19 related topics of interest amongst Moroccan social media users and sentiment/emotion analysis to gain insights into their reactions to various impactful events. The collected data consists of COVID-19 related comments posted on Twitter, Facebook and Youtube and on the websites of two popular online news outlets in Morocco (Hespress and Hibapress) throughout the year 2020. The comments are expressed in Moroccan Dialect (MD) or Modern Standard Arabic (MSA). To perform topic modeling and sentiment classification, we built a first Universal Language Model for the Moroccan Dialect (MD-ULM) using available corpora, which we have fine-tuned using our COVID-19 dataset. We show that our method significantly outperforms classical machine learning classification methods in Topic Modeling, Emotion Recognition and Polar Sentiment Analysis. To provide real-time infoveillance of these sentiments, we developed an online platform to automate the execution of the different processes, and in particular regular data collection. This platform is meant to be a decision-making assistance tool for COVID-19 mitigation and management in Morocco